Twin-delayed deep deterministic policy gradient algorithm for the energy management of microgrids

D. Domínguez-Barbero, J. García-González, M.A. Sanz-Bobi

Engineering Applications of Artificial Intelligence Vol. 125, pp. 106693-1 - 106693-11

Resumen:

The microgrid market is growing significantly due to several drivers, such as the need to lower greenhouse gas emissions by integrating higher shares of distributed renewable energy sources, falling costs of microgrid components, the need for more reliable power supply infrastructures, and new off-grid solutions to foster electricity access in developing economies. Coordinated management of the microgrid components is crucial for their effectiveness, and this can be very challenging when hosting solar or wind generation. This paper studies the energy management problem of a microgrid based on reinforcement learning algorithms. The advantage of using these algorithms against other optimization and machine learning techniques is that they do not need past experiences to learn a strategy. The learning is based on trial and error experiences, which facilitates its easy implementation to other microgrids while demonstrating their facility to be applied in real cases. In particular, this paper proposes an implementation for an Energy Management System (EMS) in microgrids using the Twin-Delayed Deep Deterministic Policy Gradient (TD3) algorithm. Moreover, it compares the proposed algorithm with the Deep Q-Network (DQN). This comparison evaluates the improvement over exploiting the continuous nature of the decision variables against a discretization of the same since the DQN cannot make actions over a continuous space.

Palabras Clave: Continuous action space; Deep reinforcement learning; Energy management system; Microgrids; TD3

Índice de impacto JCR y cuartil WoS: 8,000 - Q1 (2022)

Referencia DOI: https://doi.org/10.1016/j.engappai.2023.106693

Publicado en papel: Octubre 2023.

Publicado on-line: Julio 2023.

Cita:
D. Domínguez-Barbero, J. García-González, M.A. Sanz-Bobi, Twin-delayed deep deterministic policy gradient algorithm for the energy management of microgrids. Engineering Applications of Artificial Intelligence. Vol. 125, pp. 106693-1 - 106693-11, Octubre 2023. [Online: Julio 2023]